Context and linking in retrieval from personal digital archives

Advances in digital capture and storage technologies mean that it is now possible to capture and store one’s entire life experiences in personal digital archives. These vast personal archives (or Human Digital Memories (HDMs)) pose new challenges and opportunities for the research community, not the least of which is developing effective means of retrieval from HDMs. Personal archive retrieval research is still in its infancy and there is much scope for novel research. My PhD proposes to develop effective HDM retrieval algorithms by combining rich sources of context associated with items, such as location and people present data, with information obtained by linking HDM items in novel ways

Searching Heterogeneous Human Digital Memory Archives

Advances in digital storage technologies mean that vast digital archives of ones personal life experiences can now be generated. These personal archives (Human Digital Memories (HDMs)) can contain many types of data in various media created or accessed by the individual. These archives are of little benefit if an individual cannot locate and retrieve significant items from them. Existing search techniques are not sufficient for retrieval of items from these new unstructured spaces. This research proposes to develop effective HDM search by the integration of rich sources of context data, such as biometric information, with items in the HDM as a method to aid effective retrieval in this new domain

The information retrieval challenge of human digital memories

Today people are storing increasing amounts of personal information in digital format. While storage of such information is becoming straight forward, retrieval from the vast personal archives that this is creating poses significant challenges. Existing retrieval techniques are good at retrieving from non-personal spaces, such as the World Wide Web. However they are not sufficient for retrieval of items from these new unstructured spaces which contain items that are personal to the individual, and of which the user has personal memories and with which has had previous interaction. We believe that there are new and exciting possibilities for retrieval from personal archives. Memory cues act as triggers for individuals in the remembering process, a better understanding of memory cues will enable us to design new and effective retrieval algorithms and systems for personal archives. Context data, such as time and location, is already proving to play a key part in this special retrieval domain, for example for searching personal photo archives, we believe there are many other rich sources of context that can be exploited for retrieval from personal archives

Examining the utility of affective response in search of personal lifelogs

Personal lifelog archives contain digital records captured from an individual’s daily life, for example emails, documents edited, webpages downloaded and photographs taken. While capturing this information is becoming increasingly easy, subsequently locating interesting items from within these archives is a significant challenge. One potential source of information to identify items of importance to an individual is their affective state during the capture of the information. The strength of an individual’s affective response to their current situation can often be gauged from their physiological response. For this study we explored the utility of the following biometric features to indicate significant items: galvanic skin response (GSR), heart rate (HR) and skin temperature (ST). Significant or important events tend to raise an individual’s arousal level, causing a measurable biometric response. We examined the utility of using biometric response to identify significant items and for re-ranking traditional information retrieval (IR) result sets. Results obtained indicate that skin temperature is most useful for extracting interesting items from personal archives containing passively captured images, computer activity and SMS messages

An exploration of the utility of GSR in locating events from personal lifelogs for reflection

Digital personal lifelogs (PLs) enable many artifacts from a person’s life to be automatically stored in a digital archive. These data sets can contain a wealth of potentially valuable information describing events from an individual’s life. A key challenge for lifelog technologies is how to develop scenarios and applications which enable people to interact with these vast heterogeneous data sources in a meaningful way. One of the areas where individuals can gain from interacting with lifelog records of their life is in the process of self reflection. To date little attention has been given to applications which automatically extract content from lifelogs to support self reflection using lifelog content. One of the significant issues with reflection from lifelogs is discerning material which may be of interest in reflection from among the huge amount of available data. One way of determining the user’s engagement with their situation is measuring their biometric response associated with their arousal level. Specifically it is known that an individual’s galvanic skin response (GSR) can vary with their level of arousal. We hypothesize that situations of marked GSR variation are likely to be more significant for self reflection than other moments. We present an initial investigation, using 3 subjects’ lifelogs, of the utility of lifelog items with marked GSR for self reflection. Our results indicate that GSR records may serve as a good enabling technology for applications supporting self reflection and awareness

Venturing into the labyrinth: the information retrieval challenge of human digital memories

Advances in digital capture and storage technologies mean that it is now possible to capture and store one’s entire life experiences in a Human Digital Memory (HDM). However, these vast personal archives are of little benefit if an individual cannot locate and retrieve significant items from them. While potentially offering exciting opportunities to support a user in their activities by providing access to information stored from previous experiences, we believe that the features of HDM datasets present new research challenges for information retrieval which must be addressed if these possibilities are to be realised. Specifically we postulate that effective retrieval from HDMs must exploit the rich sources of context data which can be captured and associated with items stored within them. User’s memories of experiences stored within their memory archive will often be linked to these context features. We suggest how such contextual metadata can be exploited within the retrieval process

Considering subjects and scenarios in large-scale user-centered evaluation of a multilingual multimodal medical search system

Medical search applications can be required to service the differing information needs of multiple classes of users with varying medical knowledge levels, and language skills, as well as varying querying behaviours. The precise nature of these users' needs has to be understood to develop effective applications. Evaluation of developed search applications requires creation of holistic user-centred evaluation approaches which allow for comprehensive evaluation while being mindful of the diversity of users

Multiple multimodal mobile devices: Lessons learned from engineering lifelog solutions

For lifelogging, or the recording of one’s life history through digital means, to be successful, a range of separate multimodal mobile devices must be employed. These include smartphones such as the N95, the Microsoft SenseCam – a wearable passive photo capture device, or wearable biometric devices. Each collects a facet of the bigger picture, through, for example, personal digital photos, mobile messages and documents access history, but unfortunately, they operate independently and unaware of each other. This creates significant challenges for the practical application of these devices, the use and integration of their data and their operation by a user. In this chapter we discuss the software engineering challenges and their implications for individuals working on integration of data from multiple ubiquitous mobile devices drawing on our experiences working with such technology over the past several years for the development of integrated personal lifelogs. The chapter serves as an engineering guide to those considering working in the domain of lifelogging and more generally to those working with multiple multimodal devices and integration of their data

Applying contextual memory cues for retrieval from personal information archives

Advances in digital technologies for information capture combined with massive increases in the capacity of digital storage media mean that it is now possible to capture and store one’s entire life experiences in a Human Digital Memory (HDM). Information can be captured from a myriad of personal information devices including desktop computers, PDAs, digital cameras, video and audio recorders, and various sensors, including GPS, Bluetooth, and biometric devices. These diverse collections of personal information are potentially very valuable, but will only be so if significant information can be reliably retrieved from them. HDMs differ from traditional document collections for which existing search technologies have been developed since users may have poor recollection of contents or even the existence of stored items. Additionally HDM data is highly heterogeneous and unstructured, making it difficult to form search queries. We believe that a Personal Information Management (PIM) system which exploits the context of information capture, and potentially of earlier refinding, can be valuable in effective retrieval from an HDM. We report an investigation into how individuals perform searches of their personal information, and use the outcome of this study to develop an information retrieval (IR) framework for HDM search incorporating the context of document capture. We then describe the creation of a pilot HDM test collection, and initial experiments in retrieval from this collection. Results from these experiments indicate that use of context data can be significantly beneficial to increasing the efficient retrieval of partially recalled items from an HDM

Memory support for desktop search

The user's memory plays a very important role in desktop search. A search query with insufficiently or inaccurately recalled information may make the search dramatically less effective. In this paper, we discuss three approaches to support user’s memory during desktop search. These include extended types of well remembered search options, the use of past search queries and results, and search from similar items. We will also introduce our search system which incorporates these features